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(54) DYNAMIC IMAGE CODER AND ITS METHOD 

(57)Abstract: 

PROBLEM TO BE SOLVED: To provide a method by 
which a code amount is assigned more properly between 
picture types even when an object average bit rate is 
closer to a maximum transfer rate in a variable bit rate 
control method adopting a 1-path or 2-path method for a 
dynamic image coder. 

SOLUTION: The dynamic image coder encoding a 
dynamic image is provided with a means 23 that detects 
a code quantity generated from the dynamic image, a 
means 22 that detects a mean quantization scale of the 
dynamic image, a means 25 that detects a coding image 
characteristic of at least the dynamic image in the 
dynamic image and a motion compensation prediction 
image generated by a motion compensation prediction 
means, a 1st code quantity control means 51 that calculates a 1st assignment code quantity 
of the image to be coded next on the basis of the generated code quantity and the average 
quantization scale, a 2nd code quantity control means 52 that calculates a 2nd assigned code 
quantity placing a limit onto the 1st assigned code quantity and a means 51 that decides the 
quantization scale of the image to be coded next on the basis of the 1st and 2nd assigned 
code quantities. 
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* NOf ICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1 This document has been translated by computer. So the translation may not reflect the 
original precisely. 

2. **** shows the word which can not be translated. 

3. In the drawings, any words are not translated. 



CLAIMS 

[Claim(s)] 

[Claim 1]Video coding equipment which codes an input moving image image by motion- 
compensation-prediction means, orthogonal transformation means, quantization means, and a 
variable-length-coding means, comprising: 

A means to detect a generated code amount of each picture of said input moving image 
image. 

A means to detect a normal child-ized scale of each picture of said input moving image image. 
A means to detect the coded image characteristic of said input moving image image at least 
among motion-compensation-prediction pictures generated by said input moving image image 
and said motion-compensation-prediction means. 

The 1st code-quantity-control means that computes the 1st amount of allocation codes of a 
picture coded next from a normal child-ized scale detected by generated code amount 
detected by a means to detect said generated code amount, and a means to detect said 
normal child-ized scale. 

The 2nd code-quantity-control means that computes the 2nd amount of allocation codes for 
adding restriction to said 1st amount of allocation codes. 

A means to determine a quantizing scale of a picture coded to said next from the amount of 
allocation codes computed by the said 1st and 2nd code-quantity-control means. 

[Claim 2]ln video coding equipment indicated to Claim 1, said 2nd amount of allocation codes, 
a picture type (I picture, P picture, B picture) - a maximum of the amount of allocation codes 
being computed independently, and, When determining the actual amount of allocation codes 
in said 1st code-quantity-control means, Video coding equipment determining the 2nd amount 
of allocation codes when said 1st computed amount of allocation codes exceeds a maximum 
of the amount of allocation codes set up in said 2nd amount of allocation codes, and 
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determining said 1st amount of allocation codes as a actual amount of allocation codes when 
other. 

[Claim 3]Video coding equipment, wherein said 2nd code-quantity-control means is a code 
amount control method of a fixed bit rate in video coding equipment indicated to Claim 1 or 
Claim 2 and said 1st code-quantity-control means is a code amount control method of a 
Variable Bit Rate. 

[Claim 4]ln video coding equipment indicated to either Claim 1 thru/or Claim 3, By a 
predetermined function which has a means to compute picture complexity from a picture 
characteristic parameter detected by a means to detect a generated code amount, a normal 
child-ized scale, and the coded image characteristic of each of said detected picture, and 
makes said picture complexity a factor. Video coding equipment changing said 2nd amount of 
allocation codes. 

[Claim 5]ln video coding equipment indicated to either Claim 1 thru/or Claim 4, It has a stream 
division means which takes out a part of output-codes sequence coded by said variable- 
length-coding means, Video coding equipment, wherein said 1st code-quantity-control means 
controls a code amount of said whole output-codes sequence and controls some code 
amounts of said output-codes sequence taken out from said stream division means by said 
2nd code-quantity-control means. 

[Claim 6]ln a video encoding method which codes an input moving image image by motion- 
compensation-prediction step, orthogonal transformation step, quantization step, and a 
variable-length-coding step, A step which detects a generated code amount of each picture of 
said input moving image image, and a step which detects a normal child-ized scale of each 
picture of said input moving image image, A step which detects the coded image characteristic 
of said input moving image image at least among motion-compensation-prediction pictures 
generated by said input moving image image and said motion-compensation-prediction step, A 
generated code amount detected by a step which detects said generated code amount, The 
1st code-quantity-control step that computes the amount of allocation codes of a picture coded 
next from a normal child-ized scale detected by a step which detects said quantizing scale, 
The 2nd code-quantity-control step that computes the 2nd amount of allocation codes for 
adding restriction to said 1st amount of allocation codes, A video encoding method provided 
with a step which determines a quantizing scale of a picture coded to said next from the 
amount of allocation codes computed at the said 1st and 2nd code-quantity-control steps. 
[Claim 7]ln a video encoding method indicated to Claim 6, said 2nd amount of allocation 
codes, a picture type (I picture, P picture, B picture) - a maximum of the amount of allocation 
codes being computed independently, and, When determining the actual amount of allocation 
codes in said 1st code-quantity-control step, A video encoding method determining the 2nd 
amount of allocation codes when said 1st computed amount of allocation codes exceeds a 
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maximum of the amount of allocation codes set up in said 2nd amount of allocation codes, and 
determining said 1st amount of allocation codes as a actual amount of allocation codes when 
other. 

[Claim 8]ln a video encoding method indicated to Claim 6 or Claim 7, a generated code 
amount of each of said detected picture, A video encoding method characterized by changing 
said 2nd amount of allocation codes by a predetermined function which has a step which 
computes picture complexity from a normal child-ized scale and a picture characteristic 
parameter detected at a step which detects the coded image characteristic, and makes said 
picture complexity a factor. 

[Claim 9]ln a video encoding method indicated to either Claim 6 thru/or Claim 8, It has a 
stream division step which takes out a part of output-codes sequence coded by said variable- 
length-coding step, A video encoding method, wherein said 1st code-quantity-control step 
controls a code amount of said whole output-codes sequence and controls some code 
amounts of said output-codes sequence taken out from said stream division means by said 
2nd code-quantity-control step. 



[Translation done.] 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1 This document has been translated by computer. So the translation may not reflect the 
original precisely. 

2.**** shows the word which can not be translated. 
3.ln the drawings, any words are not translated. 



DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Field of the lnvention]High efficiency coding of video is started, and when performing variable 
bit rate encoding especially, it is related with a suitable code quantity controller and a method 
for the same. 
[0002] 

[Description of the Prior Art]MPEG 2 is already specified as international standards of the art 
which codes video, such as a television signal, highly efficiently. MPEG 2 divides the "frame" 
picture which constitutes video into the 16x1 6-pixel block called a -macro block", The 
movement quantity called a "motion vector" to each macro block unit in time in a front or the 
back between the predetermined image comparison and coded image which left several 
frames is calculated, "Motion-compensation-prediction" art which constitutes a coded image 
from an image comparison based on this movement quantity, It is specified to the error signal 
or the coded image itself of motion compensation prediction based on the component 
engineering of two image coding of "conversion coding" art which compresses the amount of 
information using DCT (discrete cosine transform) which is a kind of orthogonal transformation. 

[0003]The example of 1 composition of the video coding equipment of the conventional MPEG 
2 is shown in drawing 7 , and an example of coding picture structure is shown in drawing 6 . 
Motion compensation prediction is consisted of by combination of three kinds of pictures like 
coding picture structure which was shown in drawing 7 , which are called I picture (formation of 
a frame inner code), P picture (forward direction prediction coding), and B picture (bidirectional 
prediction coding) and from which a prediction method differs. As shown in drawing 7 , in 
conversion coding, DCT is given by DCT device 72 to the coded image itself to the output of 
the subtractor 71 which is an error signal of motion compensation prediction with the motion- 
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compensation-prediction machine 77 in P and B picture at I picture. 

[0004]After quantization controls by the output of the code quantity controller 90 and is made 
with the quantizer 73 to the DCT coefficient obtained by this DCT device 72, Variable length 
coding is made with the variable-length-coding machine 75 with the attendant information of 
others, such as a motion vector, and it is outputted after a code sequence is memorized by the 
buffer 76 as a "bit stream." Under the present circumstances, according to the sufficiency 
degree of the buffer 76, a quantizing scale is controlled by the code quantity controller 90. On 
the other hand, it supplies now decodes [ local ] and the power coefficient of the quantizer 73 
is stored in the inverse quantization device 77 and the IDCT machine 78 by the frame memory 
81 for every block. 

[0005]Since MPEG 2 is variable length coding, the generated code amount per unit time (bit 
rate) is not constant. Then, it is possible by changing suitably the quantizing scale in the case 
of quantization with the quantizer 73 into a macro block unit to control to the necessary bit rate. 
In MPEG 2 Test Model 5, the fixed bit rate control method which makes a generated code 
amount regularity by GOP units is proposed. 

[0006]ln Test Model 5, code amount assignment which changes with picture types is 
performed. While assigning most many code amounts to I picture to which frame inner code- 
ization is performed, the quantizing scale of B picture with which a decoded image is not again 
used for prediction is increased 1 .4 times of I and P picture, So that the code amount to B 
picture may be reduced, a decoded image may assign many the part to I and P picture which 
are used for prediction and the image quality of a decoded image may become fixed between 
picture types by lessening the code amount to assign further, Optimization of the code amount 
assignment by a picture type is attained. 

[0007]The fixed bit rate control method in this Test Model 5 is an effective method to the use 
as which a fixed transfer rate is required. However, since the almost same code amount is 
assigned to every portion of a video sequence, image quality deterioration will arise, without 
giving sufficient code amount to the complicated scene containing many amount of 
information. On the other hand, when the amount of information was few simple scenes, the 
code amount became a surplus and futility arose, and to the use in which a variable transfer 
rate is possible, it was not able to be said to be the suitable rate control method like DVD- 
Video. 

[0008]The rate control method which solves the above problems is the Variable Bit Rate 
control method. The coding equipment by Variable Bit Rate control is indicated by JP.H6- 
141298,A.ln this device, first, to an input moving image image, temporary coding is performed 
and a generated code amount counts for every unit time with a fixed quantity child-ized scale. 
Next, based on the generated code amount at the time of temporary coding, the target transfer 
rate of each portion is set up so that the generated code amount of the whole input moving 
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image image may become a necessary value, and - receiving an input moving image image, 
controlling to agree in this target transfer rate - the 2nd time - it codes, and in other words, 
actual code-ization is performed. 

[0009] However, by the above-mentioned conventional example, in order to obtain an output bit 
stream, at least two coding must be performed, and for a use of which real time nature is 
required, Variable Bit Rate control of a two pass method like this device cannot be used. 
[0010]On the other hand, the Variable Bit Rate control method for coding video in real time 
mostly, i.e., the Variable Bit Rate control method of an one-pass method, exists. The coding 
equipment by the Variable Bit Rate control method of an one-pass method is indicated by 
JP,H10-164577,A at drawing 6 of said gazette, etc. 

[001 1]The example of 1 composition of the video coding equipment in this conventional 
example is shown in drawing 8 . Identical codes are attached to drawing 7 and an identical 
configuration part, and the explanation is omitted. The generated code amount in the device of 
this conventional example, supply the code amount memorized to the buffer 76 to the 
generated code amount detector 83, and according to this generated code amount detector 
83, The quantizing scale from the quantizer 73 is supplied to the normal child-ized scale 
detector 82, It asks with the picture complexity calculation machine 84 by making a product 
with the average value of the quantizing scale in the screen by this normal child-ized scale 
detector 82 into "picture complexity", Based on the rate of the present picture complexity to the 
average value of the past picture complexity, the code-quantity-control machine 74 has 
realized Variable Bit Rate control by determining the target generated code quantity of a 
screen, or a targeted amount child-ized scale. 
[0012] 

[Problem(s) to be Solved by the lnvention]However, in Variable Bit Rate control, in many 
cases, restriction by a maximum transfer rate is received. When the target average bit rate is 
smaller enough than a maximum transfer rate, it is possible to make code amount assignment 
to B picture smaller than I and P picture like Test Model 5, and to optimize the code amount 
assignment between picture types. 

[0013]lf the target average bit rate becomes close to a maximum transfer rate, the amount of 
allocation codes of I and P picture will come to receive restriction by a maximum transfer rate, 
the difference of the amount of allocation codes with B picture will contract, and, occasionally 
the amount of allocation codes will become almost the same between picture types. After the 
difference of the amount of allocation codes became small, compared with B picture, the 
image quality of I and P picture worsened relatively, and although the target average bit rate 
was high, there was a problem that image quality deterioration will be perceived by the 
difference of the image quality resulting from unsuitable code amount distribution. 
[0014]Then, in the Variable Bit Rate control method of the one pass in video coding 
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equipment, and a two pass method, an object of this invention is to provide the method of 
realizing more suitable code amount assignment between picture types, when the target 
average bit rate is close to a maximum transfer rate. 
[0015] 

[Means for Solving the Problem]ln video coding equipment provided with each means of 
motion compensation prediction, such as MPEG 2, orthogonal transformation, quantization, 
and variable length coding in this invention, The 1st code-quantity-control means that 
determines the amount of allocation codes of a picture coded next from a generated code 
amount of each picture, a means to detect a normal child-ized scale and an obtained 
generated code amount, and a normal child-ized scale, It has the 2nd code-quantity-control 
means for adding restriction to the amount of allocation codes obtained by the 1st code- 
quantity-control means for every picture type. For example, when performing code amount 
assignment by Variable Bit Rate control, by the 1st code-quantity-control means, the amount of 
allocation codes by Variable Bit Rate control is calculated, and a maximum of the amount of 
allocation codes by fixed bit rate control of the highest transfer rate is searched for by the 2nd 
code-quantity-control means. Only when the amount of allocation codes obtained by the 1st 
code-quantity-control means exceeds a maximum of the amount of allocation codes obtained 
by the 2nd code-quantity-control means, a code amount actually assigned to each picture, The 
maximum is made into the actual amount of allocation codes, and when other, let the amount 
of allocation codes obtained by the 1st code-quantity-control means be the actual amount of 
allocation codes. It becomes possible to hold code amount assignment between picture types 
the optimal, without giving many code amounts unnecessary for B picture by this, when the 
target average bit rate is close to a maximum transfer rate. 

[0016]Picture complexity produced with the above-mentioned video coding equipment by 
performing predetermined operation to a generated code amount of each picture and a product 
of a normal child-ized scale is used, By setting up a predetermined function which makes 
picture complexity a factor, multiplying a maximum of the amount of allocation codes obtained 
by the 2nd aforementioned code-quantity-control means by this function, and changing a 
maximum of the amount of allocation codes, when performing Variable Bit Rate control of an 
one-pass method, Increase of a code amount in a portion near a maximum of the amount of 
allocation codes is controlled, and it becomes possible to ease image quality change at a point 
beyond a maximum of the amount of allocation codes. 

[001 7]a case where apply the above-mentioned video coding equipment and one encoded bit 
streams are shared by two decoding systems - for example, the bit stream whole - I, P, and 
all B pictures - by ****** usual coding. When one decoding system decodes the whole bit 
stream and only I and P picture are decoded in another decoding system, Control a code 
amount of the whole bit stream by the 1st code-quantity-control means, and a code amount of 
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bit stream portions of only I and P picture is controlled by the 2nd code-quantity-control means, 
It is possible by performing code amount assignment of the 1st code-quantity-control means to 
perform coding which can be decoded in two decoding systems with one coding equipment, 
restricting a code amount of I and P picture to the amount of allocation codes obtained by the 
2nd code-quantity-control means. 
[0018] 

[Embodiment of the lnvention]The 1st working example of the video coding equipment of this 
invention is described below with drawing 1 . As shown in drawing 1 , the video coding 
equipment of this invention, and the 1st working example of the method, The subtractor 1 1 , 
DCT device 12, the quantizer 13, the variable-length-coding machine 15, the buffer 16, the 
inverse quantization device 17, the I DCT machine 18, the motion-compensation-prediction 
machine 19, the adding machine 20, the frame memory 21, the normal child-ized scale 
detector 22, the generated code amount detector 23, the picture complexity calculation 
machine 24, It comprises the picture characteristic detector 25, the VBR code-quantity-control 
machine 51 , and the CBR code-quantity-control machine 52. 

[0019]The motivation picture shall be beforehand divided into the macro block unit by the 
image block separator (not shown). Motion compensation prediction is not performed about I 
picture, but the divided motivation picture is quantized by the quantizing scale sent from the 
code-quantity-control machine 14 with the quantizer 13, after DCT of the motivation image 
block itself is sent and carried out to DCT device 12 which is a kind of a DC to AC converter 
via the subtractor 11. 

[0020]The quantized signal is changed into numerals with the variable-length-coding machine 
15, and numerals are outputted after being adjusted by the following buffer 16. On the other 
hand, local decoding is carried out with the inverse quantization device 17 and the IDCT 
machine 18, and the power coefficient of the quantizer 13 is stored in the frame memory 21 for 
every block, without adding the output of the motion-compensation-prediction machine 19 with 
the adding machine 20. 

[0021]About P and B picture, the divided motivation image and the predetermined local 
decoded image block stored in the frame memory 21 are supplied to the motion- 
compensation-prediction machine 19, Motion vector detection and a motion compensation are 
performed here, and the error image block whose estimated image block the difference 
between pixels is taken between original image blocks with the subtractor 1 1 , and is a 
difference value is sent to DCT device 12. 

[0022]Numerals are outputted after DCT of the difference value was carried out by DCT device 
12, it is changed into numerals with the variable-length-coding machine 15 like I picture after 
being quantized by the quantizing scale sent from the code-quantity-control machine 14 with 
the quantizer 13, and being adjusted by the following buffer 16 after this. 
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[G023]Said estimated image block from the motion-compensation-prediction machine 19 is 
added for every pixel by the adding machine 20, and the power coefficient of the quantizer 13 
is stored in the frame memory 21 for every block, after local decoding is carried out with the 
inverse quantization device 17 and the IDCT machine 18. About each picture, the quantizing 
scale for every macro block is sent to the normal child-ized scale detector 22 from the 
quantizer 13, the quantizing scale for one frame is added there, and the normal child-ized 
scale of one frame is computed. 

[0024]On the other hand, in the buffer 16, a generated code amount is supervised and the 
value is sent to the generated code amount detector 23. In this generated code amount 
detector 23, a generated code amount is added per frame and the generated code amount of 
one frame is detected. The normal child-ized scale and generated code amount which were 
detected about every frame are sent to the picture complexity calculation machine 24 and the 
CBR code-quantity-control machine 52, respectively. 

[0025]On the other hand, in the picture characteristic detector 25, the divided original image is 
inputted, the parameter which shows a picture characteristic to a macro block unit about each 
frame of an original image, i.e., an activity, is detected, it is added per frame, and the result is 
sent to the picture complexity calculation machine 24. 

[0026]Namely, since, as for the case of I picture, motion compensation prediction is not 
performed as for the input to the picture characteristic detector 25, Only the motivation picture 
divided into the macro block unit is inputted, the activity (ACTcur) which is a parameter which 
shows a picture characteristic to a macro block unit is detected, and it is added per frame, and 
is sent to the picture complexity calculation machine 24 as the activity ACTi of I picture. 
[0027]Although distribution of a luminance value, the difference value between pixels, etc. can 
be considered as an activity (ACTcur), other parameters may be used as long as a picture 
characteristic is shown. 

[0028]On the other hand, the input to the picture characteristic detector 25 shown in drawing 1 , 
In the case of P and B picture, the motion vector used besides the divided motivation picture 
by the error picture in the motion compensation prediction of a macro block unit or the 
difference image of the coded image and image comparison in motion vector detection, and 
motion compensation prediction is inputted from the motion-compensation-prediction machine 
19. The activity (original image) ACTcur is detected from the divided motivation picture by the 
macro block unit as well as the case of I picture. 

[0029]On the other hand, in it, an absolute value sum or the square error sum is taken, and the 
error picture in the motion compensation prediction of a macro block unit or the difference 
image of the coded image and image comparison in motion vector detection is detected as the 
prediction activity ACTpred. Between contiguity macro blocks, the absolute value of difference 
is taken for every ingredient, and the direction of the motion vector used by motion 
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compensation prediction is detected as ACTmv. 

[0030]And for every macro block, by the operation of a following formula (1), ACTmb is 
computed, and it is added by one frame and sent to the picture complexity calculation machine 
24 as the activities ACTp and ACTb of P and B picture. 
[0031] 

ACTmb=a-ACTcur+b-ACTpred+c-ACTmv (1)[0032]A picture exception changes the value of 
each constant a, b, and c according to prediction mode of a macro block (are they intra, uni- 
directional prediction, or bidirectional prediction?) etc. For example, since it is thought that a 
generated code amount increases compared with the block which predicts by being set to 
b=c=0 since it does not predict like I picture in the case of Intra, the value of a is enlarged. 
[0033]Thus, presumption of the picture complexity based more on the coding characteristic is 
attained by performing activity detection adapted to prediction mode etc. 
[0034]With the picture complexity calculation machine 24, after the multiplication of the normal 
child-ized scale and generated code amount of each frame which were supplied is carried out, 
predetermined conversion is performed to a multiplication result, and it asks as picture 
complexity (past) of each frame. Average picture complexity Xi-ave of each picture type, Xp- 
ave, and Xb-ave are computed by division of the picture complexity being done with the frame 
number of the same picture type within the period, after the value within fixed time is added 
according to a coding picture type. 

[0035]The frame number beforehand defined in front in time within the fixed time said here 
from the picture which coding just ended, For example, there is also a case of fixed frame 
numbers, such as 15 frames and 300 frames, and a frame number may increase one by one 
like [ to the picture which coding just ended from the encoding start frame ]. When not fulfilling 
the fixed time which the coded frame number defined by the case of the former fixed frame 
number, either, the frame number will increase one by one like the latter. 
[0036]Picture complexity Xk-c (k= i, p, b) of the present picture to be coded from now on, The 
activity of the present picture can be presumed by the following formula (2) from ACTk (k= i, p, 
b), the same picture complexity Xk-p (k= i, p, b) of the picture of a picture type coded just 
before, and activity ACTk-p (k= i, p, b). 
[0037] 

Xk-c= Xk-p-ACTk/ACTk-p (2)[0038]When the frame which coding of the same picture type 
ended does not exist in an initial state, it asks for the picture complexity and the activity of the 
picture of each picture type by some pictures beforehand, What is necessary is to average it 
statistically according to the occurrence frequency of average video, and just to let it be an 
initial value. 

[0039]Average picture complexity Xi-ave of each picture type, Xp-ave, and Xb-ave, Presumed 
picture complexity Xi-c of the present picture to be coded from now on, Xp-c, and Xb-c are 
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sent to the VBR code-quantity-control machine 51 , and setting out of the quantizing scale for 
Variable Bit Rate control is performed here. If the frame number of 1GOP (usually interval of I 
picture) which is PictureRate and one encoding unit about BitRate and the frame number per 
second in the target average bit rate is set to N, the amount Rave of average allocation codes 
of 1GOP will be given with a following formula (3). 
[0040] 

Rave = -(BitRate/PictureRate) N (3)[0041]lf Rave of an upper type considers it as the amount 
of required allocation codes of 1GOP at the time of average picture complexity, If the picture of 
1GOP including the present picture to be coded from now on assumes that it is equal to the 
presumed picture complexity of the present picture uniformly searched for with said picture 
complexity calculation machine 24, image quality will be given to the amount Rck (k= i, p, b) of 
required allocation codes of 1GOP uniformly required for a ****** case with a following formula 

(4) . 
[0042] 

Rck= Rave-Xk-c/Xk-ave (4)[0043]By assigning Rck (k= i, p, b) of an upper type suitable for 
each picture of 1 GOP, the target code quantity in the 1st code-quantity-control means of the 
present picture to be coded from now on is computed. Although the target-code-quantity 
allocation method of MPEG 2 Test Model 5 is listed to below as an example, methods other 
than this may be used. 

[0044]The setting-out ratio of the quantizing scale of P contained in 1GOP, P [ as opposed to 
Np, Nb, and I picture for the frame number of B picture ], and B picture is set to Kp and Kb. At 
this time, the target assignment code amounts Ti, Tp, and Tb of each picture type are given 
with the following formula (5), (6), and (7). MAX [a and b] - either a or b - the operation which 
chooses the larger one is shown. Xi, Xp, and Xb are the picture complexity (product of the 
normal child-ized scale of the picture concerned, and a generated code amount) of the picture 
coded immediately before here. 
[0045] 

(I picture) Ti= MAX [A and B] A = Rc/(1+N P -x P /(xi-Kp) +Nb-xb/(xi-Kb)) B = BitRate/(8 and PictureRate) 

(5) [0046] 

(P picture) Tp= MAX [C and D] C = Rc/(N P+ Nb-K P -xb/(Kb-x P )) D = BitRate/(8 and PictureRate) 

(6) [0047] 

(B picture) Tb= MAX [E and F] E = Rc/(N P+ N P -Kb-xp/(K P -Kb)) F = BitRate/(8 and PictureRate) 

(7) [0048]On the other hand with the CBR code-quantity-control machine 52, the normal child- 
ized scale of a frame unit, A generated code amount is inputted and the picture complexity Xi 
of the picture coded from both product just before, Xp, and Xb are calculated, Target 
assignment code amount Ti-max of each picture type in case BitRate is the highest transfer 
rate (BitRateMax), Tp-max, and Tb-max are calculated like the target assignment code 
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amounts Ti, Tp, and Tb in the 1st code-quantity-control means. Here, amount Rav-max of 
average allocation codes of 1GOP is common to each picture, and is given with a following 
formula (8). 
[0049] 

Rav-max= -(BitRateMax/PictureRate) N (8) Therefore, [0050] 

(I picture) Ti-max= MAX [A and B] A =Rav-max /(1+N P -x P /(xi-K P ) + N b-xb/(xi-Kt>)) B = BitRateMax/(8 
and PictureRate) 

(9) [0051] 

(P picture) Tp-max = MAX [C and D] C =Rav-max /(N P+ Nb-K P -xb/(Kb-x P )) D = BitRateMax/(8 and 
PictureRate) 

(10) [0052] 

(B picture) Tb-max = MAX [E and F] E = Rav-max/(N P+ N P -Kb-x P /(K P -Kb)) F = BitRateMax/(8 and 
PictureRate) 

(1 1) [0053]Ti-max of an upper type, Tp-max, and Tb-max, Namely, it is a maximum of the 
target assignment code amount in the 2nd code-quantity-control means, These values are sent 
to the VBR code-quantity-control machine 51, about the picture type concerned of the above 
mentioned picture of the present when it codes, a limiting circuit is applied with the value of Ti- 
max, Tp-max, and Tb-max, and the target assignment code amount of the present picture 
determines the value of above mentioned Ti, Tp, and Tb. 

[0054]Based on the target assignment code amount determined as mentioned above and the 
generated code amount of each macro block detected with the buffer 16, the method of MPEG 
2 Test Model 5 is used, and the quantizing scale of each macro block is determined. 
[0055]From the picture characteristic detector 25, the activity ACTcur of each macro block is 
sent also to the code-quantity-control machine 51 , Although used for the adaptive-quantization 
control which changes the quantizing scale of each macro block based on the activity in MPEG 
2 Test Model 5, it is not necessary to perform this adaptive-quantization control. The 
quantizing scale of each macro block may be determined by a completely different method 
from this. 

[0056]The quantizing scale of each macro block outputted from the code-quantity-control 
machine 51 is sent to the quantizer 13, Numerals are outputted, after variable length coding of 
the present picture (the divided original image after DCT or the error image block of motion 
compensation prediction) is quantized and carried out with this quantizing scale and it is 
adjusted with the buffer 1 6. 

[0057]The generated code amount supervised with the quantizing scale for every macro block 
and the buffer 16 is sent to the normal child-ized scale detector 22 and the generated code 
amount detector 23, and is used for the code quantity control of the following picture, 
respectively. 
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[0058]ln the upper explanation, amount Rav-max of average allocation codes of 1GOP in the 
CBR code-quantity-control machine 52 was calculated as a code amount to which it is simply 
assigned by 1 GOP at the time of the highest transfer rate (BitRateMax).On the other hand, if 
presumed picture complexity Xk-c of a predetermined function like drawing 2 (a) which makes 
a factor presumed picture complexity Xk-c (k= i, p, b) of the present picture to be coded from 
now on, for example, the present picture, increases, the value will set up the function f (Xk-c) 
which approaches 1 infinite. 

[0059]By using Rav-max' of the following formula (12) which carried out the multiplication of 
this function for every picture type instead of Rav-max, As shown in drawing 2 (b), press down 
gradually the generated code amount in the case of being close to the highest transfer rate of 
each picture type, and. The image quality deterioration of the picture type concerned produced 
from the relation between picture complexity and the amount of allocation codes becoming 
discontinuous at the point beyond the highest transfer rate becomes possible [ also pressing 
down the problem which becomes remarkable ]. 
[0060] 

Rav-max' -- =f(Xk-c), (BitRateMax/PictureRate), andN - the function (k=i, p, b) (12) which 
shows drawing 2 (a) f (Xk-c) here[0061]The 2nd working example of the video coding 
equipment of this invention is described with drawi ng 3 below. In the 2nd working example, it is 
a case where this invention is applied to the Variable Bit Rate code quantity control of a two 
pass method. The fundamental coding portion until it is changed into numerals with an original 
image input - the variable-length-coding machine 15 is the same as that of the 1st working 
example. Since encoding operation is performed twice (or more than it), temporary coding is 
performed first and the 2nd coding is performed based on the result of the generated code 
amount about one picture, it differs greatly that there is a portion which is different in both 
encoding operation. 

[0062]ln the 1st coding, the quantizing scale sent to the quantizer 13 is not sent from the VBR 
code-quantity-control machine 51 , but a fixed value (values, such as 6 and 8) is sent from the 
temporary coding quantization set scale machine 56 via switch SW1 , and, thereby, 
quantization of a fixed value is performed. And the bit stream after variable length coding was 
performed in the variable-length-coding machine 15 is not sent to the buffer 16 for outputting it 
outside, but it is sent to the temporary coding generated code amount detector 53 via switch 
SW2, and the generated code amount of each picture in the 1st coding is detected. 
[0063]A temporary transfer rate is computed by a generated code amount being sent to the 
temporary transfer rate memory 54 from the temporary coding generated code amount 
detector 53 one by one, and being added for every prescribed period. This operation is 
performed until coding of one image sequence is completed, and the temporary transfer rate 
for every prescribed period is accumulated in the temporary transfer rate memory 54. 
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[0064]After the 1st coding is completed, the temporary generated code amount or average 
temporary transfer rate of the whole image sequence is computed, this value and the 
temporary transfer rate for every prescribed period are sent to the target transfer rate 
calculation machine 55, and the target transfer rate for every [ in the 2nd coding (actual-code- 
izing) ] prescribed period is computed. 

[0065]The relation of the temporary transfer rate Rt for every prescribed period and target 
transfer rate R of the 2nd coding in the 1st coding sets up the predetermined function 
beforehand. For example, the following function as shown in (13) can be considered. 
[0066] 

R=a- (Rt) (13) b (a and b are a constant, a> 0, and 0< b<1)[0067]The 1st temporary coding is 
completed, and if the target transfer rate of the coding which is the 2nd time is determined, the 
2nd coding (actual-code-izing) will be started according to the target transfer rate. In the 2nd 
coding, the value from which the quantizing scale sent to the quantizer 13 was obtained with 
the VBR code-quantity-control machine 51 is sent. 

[0068]The normal child-ized scale of each frame detected with the VBR code-quantity-control 
machine 51 here with the normal child-ized scale detector 22 and the generated code amount 
detector 23, and a generated code amount, From the target transfer rate for every prescribed 
period computed from the temporary encoded result with the target transfer rate calculation 
machine 55, the target assignment code amount of the picture to be coded from now on is 
calculated. 

[0069]On the other hand, it is inputted into the VBR code-quantity-control machine 51 , a 
limiting circuit is applied to said target assignment code amount, and a target assignment code 
amount also determines the maximum of the target assignment (it can set for 2nd code- 
quantity-control means) code amount computed with the CBR code-quantity-control machine 
52. The maximum of the target assignment code amount in the CBR code-quantity-control 
machine 52 is the same as that of (based on Rav-max) Ti-max in the 1st working example, Tp- 
max, and Tb-max. 

[0070]The quantizing scale of each macro block is determined using the method of MPEG 2 
Test Model 5 grade like the 1st working example based on the target assignment code amount 
determined as mentioned above and the generated code amount of each macro block 
detected with the buffer 16. 

[0071]Thus, the quantizing scale of each determined macro block is sent to the quantizer 13, 
and variable length coding of the picture (the divided original image after DCT or the error 
image block of motion compensation prediction) to be coded from now on is quantized and 
carried out with this quantizing scale. 

[0072] Numerals are outputted after being adjusted by the target transfer rate for every 
prescribed period which the bit stream generated here was supplied to the buffer 16 by the 2nd 
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coding, and was computed with the target transfer rate calculation machine 55 here. The 
generated code amount supervised with the quantizing scale for every macro block and the 
buffer 16 is sent to the normal child-ized scale detector 22 and the generated code amount 
detector 23, and is used for the code quantity control of the following picture, respectively. 
[0073]The 3rd working example of the video coding equipment of this invention is described 
with drawing 4 and drawing 5 below. Although the above working example [ 1st and 2nd ] is 
the cases where this invention is applied to Variable Bit Rate code quantity control, this 
invention is not limited to it but broad application is possible for it. Like the video coding 
equipment shown in drawing 4, the encoded bit streams of the output of the buffer 16 are 
divided into two with the stream separator 59. The output of one of these outputs the whole 
encoded bit streams which use I like drawing 5 , P, and all B pictures, and another side outputs 
the encoded bit streams which use only I and P picture before long. 
[0074]The output of only I and P picture shall be a header conversion machine which is not 
illustrated here, and the parameter of a header part, etc. shall be rewritten by the suitable 
value. The code-quantity-control machine 1 (51 A) of drawing 4 controls the code amount of the 
whole bit stream, and the code-quantity-control machine 2 (52A) controls the code amount of 
the bit stream of only I and P picture. 

[0075]Here, both average bit rate shall be set up so that the amount of average allocation 
codes of each picture type may serve as a comparatively near value with the code-quantity- 
control machine 1 (51 A) and the code-quantity-control machine 2 (52A). In order to satisfy the 
code quantity control of two bit streams simultaneously, the code amount quota result of I in 
the code-quantity-control machine 2 (52A) and P picture is sent to the code-quantity-control 
machine 1 (51 A), With the code-quantity-control machine 1 (51 A), to I and P picture, I as a 
result of the code-quantity-control machine 2 (52A) and code amount assignment of P picture 
are applied as it is, code amount assignment is newly performed about B picture, and control 
of the code-quantity-control machine 1 (51 A) is realized. 

[0076]Thereby, by the whole bit stream, while control with the code-quantity-control machine 1 
(51 A) is performed, when I of a bit stream and P picture portion are taken out with the stream 
separator 59, the bit stream controlled by the code-quantity-control machine 2 (52A) can be 
obtained. 

[0077]When the bit stream of only I and P picture does not need to be outputted with the 
stream separator 59, in the code-quantity-control machine 1 (51A), the usual code amount 
assignment is performed about I, P, and B pictures of each, without performing code amount 
assignment with the code-quantity-control machine 2 (52A). When not performing code 
amount assignment with the code-quantity-control machine 2 (52A), the signal which shows it 
is sent to the stream separator 59, and an output is stopped for the bit stream of only I and P 
picture there. 
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[0078]ln the code-quantity-control gestalt in which not only working example of drawing 4 but 
this invention has two code-quantity-control machines, as opposed to the code-quantity-control 
machine 51 A of a main direction - ** - by adding restriction by the code amount quota result 
for every picture type of the code-quantity-control machine 52A of a direction, Rationalization 
of code amount assignment of each picture type can be attained, or the video coding 
equipment corresponding to two decoding systems can be realized to one encoded bit 
streams. 
[0079] 

[Effect of the lnvention]The 1st code-quantity-control means that determines the amount of 
allocation codes of a coded image as mentioned above according to this invention, It has the 
2nd code-quantity-control means that adds restriction to code amount assignment of the 1st 
code-quantity-control means, In Variable Bit Rate control, by the 1st code-quantity-control 
means For example, Variable Bit Rate control, By the 2nd code-quantity-control means, 
calculate the amount of allocation codes by fixed bit rate control of the highest transfer rate, 
and the actual amount of allocation codes, Only when the amount of allocation codes obtained 
by the 1st code-quantity-control means exceeds the amount of allocation codes obtained by 
the 2nd code-quantity-control means, the amount of allocation codes obtained by the 2nd 
code-quantity-control means is applied, and when other, the amount of allocation codes 
obtained by the 1st code-quantity-control means is applied. It becomes possible to hold the 
code amount assignment between picture types the optimal, without giving many code 
amounts unnecessary for B picture by this, when the target average bit rate is close to a 
maximum transfer rate. 

[0080]When control using the picture complexity produced by Variable Bit Rate control of an 
one-pass method by performing predetermined operation to the generated code amount of 
each picture and the product of a normal child-ized scale is performed, By setting up the 
predetermined function which makes picture complexity a factor, multiplying the maximum of 
the amount of allocation codes obtained by the 2nd aforementioned code-quantity-control 
means by this function, and changing the maximum of the amount of allocation codes, 
Increase of the code amount in the portion near the maximum of the amount of allocation 
codes is controlled, and it becomes possible to ease image quality change at the point beyond 
the maximum of the amount of allocation codes. 

[0081]Also in the case where apply the above-mentioned video coding equipment and one 
encoded bit streams are shared by two decoding systems, By restricting code amount 
assignment of the 1st code-quantity-control means by code amount assignment obtained by 
the 2nd code-quantity-control means, it becomes possible to perform the coding which can be 
decoded in two decoding systems with one coding equipment. 
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IThis document has been translated by computer. So the translation may not reflect the 
original precisely. 

2.**** shows the word which can not be translated. 
3.ln the drawings, any words are not translated. 



TECHNICAL FIELD 

[Field of the InventionJHigh efficiency coding of video is started, and when performing variable 
bit rate encoding especially, it is related with a suitable code quantity controller and a method 
for the same. 



[Translation done.] 



http://www4.ipdl.inpit.go.jp/cgi-bin/tran_web_cgi_ejje?atw_u=http7o3A7o2... 5/13/2010 



JP,2001-028753,A [PRIOR ART] 



Page 1 of 3 



* NOTICES* 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the 
original precisely. 

2. **** shows the word which can not be translated. 
3.ln the drawings, any words are not translated. 



PRIOR ART 

[Description of the Prior Art]MPEG 2 is already specified as international standards of the art 
which codes video, such as a television signal, highly efficiently. MPEG 2 divides the -frame- 
picture which constitutes video into the 16x1 6-pixel block called a -macro block-, The 
movement quantity called a "motion vector" to each macro block unit in time in a front or the 
back between the predetermined image comparison and coded image which left several 
frames is calculated, "Motion-compensation-prediction" art which constitutes a coded image 
from an image comparison based on this movement quantity, It is specified to the error signal 
or the coded image itself of motion compensation prediction based on the component 
engineering of two image coding of "conversion coding" art which compresses the amount of 
information using DCT (discrete cosine transform) which is a kind of orthogonal transformation. 

[0003]The example of 1 composition of the video coding equipment of the conventional MPEG 
2 is shown in drawing 7 , and an example of coding picture structure is shown in drawing 6 . 
Motion compensation prediction is consisted of by combination of three kinds of pictures like 
coding picture structure which was shown in drawing 7 , which are called I picture (formation of 
a frame inner code), P picture (forward direction prediction coding), and B picture (bidirectional 
prediction coding) and from which a prediction method differs. As shown in drawing 7 , in 
conversion coding, DCT is given by DCT device 72 to the coded image itself to the output of 
the subtractor 71 which is an error signal of motion compensation prediction with the motion- 
compensation-prediction machine 77 in P and B picture at I picture. 
[0004]After quantization controls by the output of the code quantity controller 90 and is made 
with the quantizer 73 to the DCT coefficient obtained by this DCT device 72, Variable length 
coding is made with the variable-length-coding machine 75 with the attendant information of 
others, such as a motion vector, and it is outputted after a code sequence is memorized by the 
buffer 76 as a "bit stream." Under the present circumstances, according to the sufficiency 
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degree of the buffer 76, a quantizing scale is controlled by the code quantity controller 90. On 
the other hand, it supplies now decodes [ local ] and the power coefficient of the quantizer 73 
is stored in the inverse quantization device 77 and the IDCT machine 78 by the frame memory 
81 for every block. 

[0005]Since MPEG 2 is variable length coding, the generated code amount per unit time (bit 
rate) is not constant. Then, it is possible by changing suitably the quantizing scale in the case 
of quantization with the quantizer 73 into a macro block unit to control to the necessary bit rate. 
In MPEG 2 Test Model 5, the fixed bit rate control method which makes a generated code 
amount regularity by GOP units is proposed. 

[0006]ln Test Model 5, code amount assignment which changes with picture types is 
performed. While assigning most many code amounts to I picture to which frame inner code- 
ization is performed, the quantizing scale of B picture with which a decoded image is not again 
used for prediction is increased 1 .4 times of I and P picture, So that the code amount to B 
picture may be reduced, a decoded image may assign many the part to I and P picture which 
are used for prediction and the image quality of a decoded image may become fixed between 
picture types by lessening the code amount to assign further, Optimization of the code amount 
assignment by a picture type is attained. 

[0007]The fixed bit rate control method in this Test Model 5 is an effective method to the use 
as which a fixed transfer rate is required. However, since the almost same code amount is 
assigned to every portion of a video sequence, image quality deterioration will arise, without 
giving sufficient code amount to the complicated scene containing many amount of 
information. On the other hand, when the amount of information was few simple scenes, the 
code amount became a surplus and futility arose, and to the use in which a variable transfer 
rate is possible, it was not able to be said to be the suitable rate control method like DVD- 
Video. 

[0008]The rate control method which solves the above problems is the Variable Bit Rate 
control method. The coding equipment by Variable Bit Rate control is indicated by JP.H6- 
141298,A.ln this device, first, to an input moving image image, temporary coding is performed 
and a generated code amount counts for every unit time with a fixed quantity child-ized scale. 
Next, based on the generated code amount at the time of temporary coding, the target transfer 
rate of each portion is set up so that the generated code amount of the whole input moving 
image image may become a necessary value, and - receiving an input moving image image, 
controlling to agree in this target transfer rate — the 2nd time - it codes, and in other words, 
actual code-ization is performed. 

[0009] However, by the above-mentioned conventional example, in order to obtain an output bit 
stream, at least two coding must be performed, and for a use of which real time nature is 
required, Variable Bit Rate control of a two pass method like this device cannot be used. 
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[0010]On the other hand, the Variable Bit Rate control method for coding video in real time 
mostly, i.e., the Variable Bit Rate control method of an one-pass method, exists. The coding 
equipment by the Variable Bit Rate control method of an one-pass method is indicated by 
JP,H10-164577,A at drawing 6 of said gazette, etc. 

[0011]The example of 1 composition of the video coding equipment in this conventional 
example is shown in drawing 8 . Identical codes are attached to drawing 7 and an identical 
configuration part, and the explanation is omitted. The generated code amount in the device of 
this conventional example, supply the code amount memorized to the buffer 76 to the 
generated code amount detector 83, and according to this generated code amount detector 
83, The quantizing scale from the quantizer 73 is supplied to the normal child-ized scale 
detector 82, It asks with the picture complexity calculation machine 84 by making a product 
with the average value of the quantizing scale in the screen by this normal child-ized scale 
detector 82 into "picture complexity", Based on the rate of the present picture complexity to the 
average value of the past picture complexity, the code-quantity-control machine 74 has 
realized Variable Bit Rate control by determining the target generated code quantity of a 
screen, or a targeted amount child-ized scale. 
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EFFECT OF THE INVENTION 

[Effect of the lnvention]The 1st code-quantity-control means that determines the amount of 
allocation codes of a coded image as mentioned above according to this invention, It has the 
2nd code-quantity-control means that adds restriction to code amount assignment of the 1st 
code-quantity-control means, In Variable Bit Rate control, by the 1st code-quantity-control 
means For example, Variable Bit Rate control, By the 2nd code-quantity-control means, 
calculate the amount of allocation codes by fixed bit rate control of the highest transfer rate, 
and the actual amount of allocation codes, Only when the amount of allocation codes obtained 
by the 1st code-quantity-control means exceeds the amount of allocation codes obtained by 
the 2nd code-quantity-control means, the amount of allocation codes obtained by the 2nd 
code-quantity-control means is applied, and when other, the amount of allocation codes 
obtained by the 1st code-quantity-control means is applied. It becomes possible to hold the 
code amount assignment between picture types the optimal, without giving many code 
amounts unnecessary for B picture by this, when the target average bit rate is close to a 
maximum transfer rate. 

[0080]When control using the picture complexity produced by Variable Bit Rate control of an 
one-pass method by performing predetermined operation to the generated code amount of 
each picture and the product of a normal child-ized scale is performed, By setting up the 
predetermined function which makes picture complexity a factor, multiplying the maximum of 
the amount of allocation codes obtained by the 2nd aforementioned code-quantity-control 
means by this function, and changing the maximum of the amount of allocation codes, 
Increase of the code amount in the portion near the maximum of the amount of allocation 
codes is controlled, and it becomes possible to ease image quality change at the point beyond 
the maximum of the amount of allocation codes. 

[0081]Also in the case where apply the above-mentioned video coding equipment and one 
encoded bit streams are shared by two decoding systems, By restricting code amount 
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assignment of the 1st code-quantity-control means by code amount assignment obtained by 
the 2nd code-quantity-control means, it becomes possible to perform the coding which can be 
decoded in two decoding systems with one coding equipment. 
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TECHNICAL PROBLEM 

[Problem(s) to be Solved by the lnvention]However, in Variable Bit Rate control, in many 
cases, restriction by a maximum transfer rate is received. When the target average bit rate is 
smaller enough than a maximum transfer rate, it is possible to make code amount assignment 
to B picture smaller than I and P picture like Test Model 5, and to optimize the code amount 
assignment between picture types. 

[001 3]lf the target average bit rate becomes close to a maximum transfer rate, the amount of 
allocation codes of I and P picture will come to receive restriction by a maximum transfer rate, 
the difference of the amount of allocation codes with B picture will contract, and, occasionally 
the amount of allocation codes will become almost the same between picture types. After the 
difference of the amount of allocation codes became small, compared with B picture, the 
image quality of I and P picture worsened relatively, and although the target average bit rate 
was high, there was a problem that image quality deterioration will be perceived by the 
difference of the image quality resulting from unsuitable code amount distribution. 
[0014]Then, in the Variable Bit Rate control method of the one pass in video coding 
equipment, and a two pass method, an object of this invention is to provide the method of 
realizing more suitable code amount assignment between picture types, when the target 
average bit rate is close to a maximum transfer rate. 
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MEANS 

[Means for Solving the Problem]ln video coding equipment provided with each means of 
motion compensation prediction, such as MPEG 2, orthogonal transformation, quantization, 
and variable length coding in this invention, The 1st code-quantity-control means that 
determines the amount of allocation codes of a picture coded next from a generated code 
amount of each picture, a means to detect a normal child-ized scale and an obtained 
generated code amount, and a normal child-ized scale, It has the 2nd code-quantity-control 
means for adding restriction to the amount of allocation codes obtained by the 1st code- 
quantity-control means for every picture type. For example, when performing code amount 
assignment by Variable Bit Rate control, by the 1st code-quantity-control means, the amount of 
allocation codes by Variable Bit Rate control is calculated, and a maximum of the amount of 
allocation codes by fixed bit rate control of the highest transfer rate is searched for by the 2nd 
code-quantity-control means. Only when the amount of allocation codes obtained by the 1st 
code-quantity-control means exceeds a maximum of the amount of allocation codes obtained 
by the 2nd code-quantity-control means, a code amount actually assigned to each picture, The 
maximum is made into the actual amount of allocation codes, and when other, let the amount 
of allocation codes obtained by the 1st code-quantity-control means be the actual amount of 
allocation codes. It becomes possible to hold code amount assignment between picture types 
the optimal, without giving many code amounts unnecessary for B picture by this, when the 
target average bit rate is close to a maximum transfer rate. 

[0016]Picture complexity produced with the above-mentioned video coding equipment by 
performing predetermined operation to a generated code amount of each picture and a product 
of a normal child-ized scale is used, By setting up a predetermined function which makes 
picture complexity a factor, multiplying a maximum of the amount of allocation codes obtained 
by the 2nd aforementioned code-quantity-control means by this function, and changing a 
maximum of the amount of allocation codes, when performing Variable Bit Rate control of an 
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one-pass method, Increase of a code amount in a portion near a maximum of the amount of 
allocation codes is controlled, and it becomes possible to ease image quality change at a point 
beyond a maximum of the amount of allocation codes. 

[001 7]a case where apply the above-mentioned video coding equipment and one encoded bit 
streams are shared by two decoding systems -- for example, the bit stream whole - I, P, and 
all B pictures - by ****** usual coding. When one decoding system decodes the whole bit 
stream and only I and P picture are decoded in another decoding system, Control a code 
amount of the whole bit stream by the 1st code-quantity-control means, and a code amount of 
bit stream portions of only I and P picture is controlled by the 2nd code-quantity-control means, 
It is possible by performing code amount assignment of the 1st code-quantity-control means to 
perform coding which can be decoded in two decoding systems with one coding equipment, 
restricting a code amount of I and P picture to the amount of allocation codes obtained by the 
2nd code-quantity-control means. 
[0018] 

[Embodiment of the lnvention]The 1st working example of the video coding equipment of this 
invention is described below with drawing 1 . As shown in drawing 1 , the video coding 
equipment of this invention, and the 1st working example of the method, The subtractor 1 1 , 
DCT device 12, the quantizer 13, the variable-length-coding machine 15, the buffer 16, the 
inverse quantization device 17, the IDCT machine 18, the motion-compensation-prediction 
machine 19, the adding machine 20, the frame memory 21, the normal child-ized scale 
detector 22, the generated code amount detector 23, the picture complexity calculation 
machine 24, It comprises the picture characteristic detector 25, the VBR code-quantity-control 
machine 51 , and the CBR code-quantity-control machine 52. 

[0019]The motivation picture shall be beforehand divided into the macro block unit by the 
image block separator (not shown). Motion compensation prediction is not performed about I 
picture, but the divided motivation picture is quantized by the quantizing scale sent from the 
code-quantity-control machine 14 with the quantizer 13, after DCT of the motivation image 
block itself is sent and carried out to DCT device 12 which is a kind of a DC to AC converter 
via the subtractor 1 1 . 

[0020]The quantized signal is changed into numerals with the variable-length-coding machine 
15, and numerals are outputted after being adjusted by the following buffer 16. On the other 
hand, local decoding is carried out with the inverse quantization device 17 and the IDCT 
machine 18, and the power coefficient of the quantizer 13 is stored in the frame memory 21 for 
every block, without adding the output of the motion-compensation-prediction machine 19 with 
the adding machine 20. 

[0021]About P and B picture, the divided motivation image and the predetermined local 
decoded image block stored in the frame memory 21 are supplied to the motion- 
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compensation-prediction machine 19, Motion vector detection and a motion compensation are 
performed here, and the error image block whose estimated image block the difference 
between pixels is taken between original image blocks with the subtractor 1 1 , and is a 
difference value is sent to DCT device 12. 

[0022]Numerals are outputted after DCT of the difference value was carried out by DCT device 
12, it is changed into numerals with the variable-length-coding machine 15 like I picture after 
being quantized by the quantizing scale sent from the code-quantity-control machine 14 with 
the quantizer 13, and being adjusted by the following buffer 16 after this. 
[0023]Said estimated image block from the motion-compensation-prediction machine 19 is 
added for every pixel by the adding machine 20, and the power coefficient of the quantizer 13 
is stored in the frame memory 21 for every block, after local decoding is carried out with the 
inverse quantization device 17 and the IDCT machine 18. About each picture, the quantizing 
scale for every macro block is sent to the normal child-ized scale detector 22 from the 
quantizer 13, the quantizing scale for one frame is added there, and the normal child-ized 
scale of one frame is computed. 

[0024]On the other hand, in the buffer 16, a generated code amount is supervised and the 
value is sent to the generated code amount detector 23. In this generated code amount 
detector 23, a generated code amount is added per frame and the generated code amount of 
one frame is detected. The normal child-ized scale and generated code amount which were 
detected about every frame are sent to the picture complexity calculation machine 24 and the 
CBR code-quantity-control machine 52, respectively. 

[0025]On the other hand, in the picture characteristic detector 25, the divided original image is 
inputted, the parameter which shows a picture characteristic to a macro block unit about each 
frame of an original image, i.e., an activity, is detected, it is added per frame, and the result is 
sent to the picture complexity calculation machine 24. 

[0026]Namely, since, as for the case of I picture, motion compensation prediction is not 
performed as for the input to the picture characteristic detector 25, Only the motivation picture 
divided into the macro block unit is inputted, the activity (ACTcur) which is a parameter which 
shows a picture characteristic to a macro block unit is detected, and it is added per frame, and 
is sent to the picture complexity calculation machine 24 as the activity ACTi of I picture. 
[0027] Although distribution of a luminance value, the difference value between pixels, etc. can 
be considered as an activity (ACTcur), other parameters may be used as long as a picture 
characteristic is shown. 

[0028]On the other hand, the input to the picture characteristic detector 25 shown in drawing 1, 
In the case of P and B picture, the motion vector used besides the divided motivation picture 
by the error picture in the motion compensation prediction of a macro block unit or the 
difference image of the coded image and image comparison in motion vector detection, and 
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motion compensation prediction is inputted from the motion-compensation-prediction machine 
19. The activity (original image) ACTcur is detected from the divided motivation picture by the 
macro block unit as well as the case of I picture. 

[0029]On the other hand, in it, an absolute value sum or the square error sum is taken, and the 
error picture in the motion compensation prediction of a macro block unit or the difference 
image of the coded image and image comparison in motion vector detection is detected as the 
prediction activity ACTpred. Between contiguity macro blocks, the absolute value of difference 
is taken for every ingredient, and the direction of the motion vector used by motion 
compensation prediction is detected as ACTmv. 

[0030]And for every macro block, by the operation of a following formula (1), ACTmb is 
computed, and it is added by one frame and sent to the picture complexity calculation machine 
24 as the activities ACTp and ACTb of P and B picture. 
[0031] 

ACTmb=a-ACTcur+b-ACTpred+c-ACTmv (1)[0032]A picture exception changes the value of 
each constant a, b, and c according to prediction mode of a macro block (are they intra, uni- 
directional prediction, or bidirectional prediction?) etc. For example, since it is thought that a 
generated code amount increases compared with the block which predicts by being set to 
b=c=0 since it does not predict like I picture in the case of Intra, the value of a is enlarged. 
[0033]Thus, presumption of the picture complexity based more on the coding characteristic is 
attained by performing activity detection adapted to prediction mode etc. 
[0034]With the picture complexity calculation machine 24, after the multiplication of the normal 
child-ized scale and generated code amount of each frame which were supplied is carried out, 
predetermined conversion is performed to a multiplication result, and it asks as picture 
complexity (past) of each frame. Average picture complexity Xi-ave of each picture type, Xp- 
ave, and Xb-ave are computed by division of the picture complexity being done with the frame 
number of the same picture type within the period, after the value within fixed time is added 
according to a coding picture type. 

[0035]The frame number beforehand defined in front in time within the fixed time said here 
from the picture which coding just ended, For example, there is also a case of fixed frame 
numbers, such as 1 5 frames and 300 frames, and a frame number may increase one by one 
like [ to the picture which coding just ended from the encoding start frame ]. When not fulfilling 
the fixed time which the coded frame number defined by the case of the former fixed frame 
number, either, the frame number will increase one by one like the latter. 
[0036]Picture complexity Xk-c (k= i, p, b) of the present picture to be coded from now on, The 
activity of the present picture can be presumed by the following formula (2) from ACTk (k= i, p, 
b), the same picture complexity Xk-p (k= i, p, b) of the picture of a picture type coded just 
before, and activity ACTk-p (k= i, p, b). 
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[0037] 

Xk-c= Xk-p-ACTk/ACTk-p (2)[0038]When the frame which coding of the same picture type 
ended does not exist in an initial state, it asks for the picture complexity and the activity of the 
picture of each picture type by some pictures beforehand, What is necessary is to average it 
statistically according to the occurrence frequency of average video, and just to let it be an 
initial value. 

[0039]Average picture complexity Xi-ave of each picture type, Xp-ave, and Xb-ave, Presumed 
picture complexity Xi-c of the present picture to be coded from now on, Xp-c, and Xb-c are 
sent to the VBR code-quantity-control machine 51, and setting out of the quantizing scale for 
Variable Bit Rate control is performed here. If the frame number of 1GOP (usually interval of I 
picture) which is PictureRate and one encoding unit about BitRate and the frame number per 
second in the target average bit rate is set to N, the amount Rave of average allocation codes 
of 1GOP will be given with a following formula (3). 
[0040] 

Rave = -(BitRate/PictureRate) N (3)[0041]lf Rave of an upper type considers it as the amount 
of required allocation codes of 1GOP at the time of average picture complexity, If the picture of 
1GOP including the present picture to be coded from now on assumes that it is equal to the 
presumed picture complexity of the present picture uniformly searched for with said picture 
complexity calculation machine 24, image quality will be given to the amount Rck (k= i, p, b) of 
required allocation codes of 1GOP uniformly required for a ****** case with a following formula 

(4) . 
[0042] 

Rck= Rave-Xk-c/Xk-ave (4)[0043]By assigning Rck (k= i, p, b) of an upper type suitable for 
each picture of 1 GOP, the target code quantity in the 1st code-quantity-control means of the 
present picture to be coded from now on is computed. Although the target-code-quantity 
allocation method of MPEG 2 Test Model 5 is listed to below as an example, methods other 
than this may be used. 

[0044]The setting-out ratio of the quantizing scale of P contained in 1GOP, P [ as opposed to 
Np, Nb, and I picture for the frame number of B picture ], and B picture is set to Kp and Kb. At 
this time, the target assignment code amounts Ti, Tp, and Tb of each picture type are given 
with the following formula (5), (6), and (7). MAX [a and b] - either a or b - the operation which 
chooses the larger one is shown. Xi, Xp, and Xb are the picture complexity (product of the 
normal child-ized scale of the picture concerned, and a generated code amount) of the picture 
coded immediately before here. 
[0045] 

(I picture) Ti= MAX [A and B] A = Rc/(1+N P -xp/(xi-Kp) +Nb-xb/(xi-Kb)) B = BitRate/(8 and PictureRate) 

(5) [0046] 
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(P picture) Tp= MAX [C and D] C = Rc/(N P+ Nb-K P -xb /(Kb-xp)) D = BitRate/(8 and PictureRate) 

(6) [0047] 

(B picture) Tb= MAX [E and F] E = Rc/(N P+ N P -Kb-x P /(x P -Kb)) F = BitRate/(8 and PictureRate) 

(7) [0048]On the other hand with the CBR code-quantity-control machine 52, the normal child- 
ized scale of a frame unit, A generated code amount is inputted and the picture complexity Xi 
of the picture coded from both product just before, Xp, and Xb are calculated, Target 
assignment code amount Ti-max of each picture type in case BitRate is the highest transfer 
rate (BitRateMax), Tp-max, and Tb-max are calculated like the target assignment code 
amounts Ti, Tp, and Tb in the 1st code-quantity-control means. Here, amount Rav-max of 
average allocation codes of 1GOP is common to each picture, and is given with a following 
formula (8). 

[0049] 

Rav-max= -(BitRateMax/PictureRate) N (8) Therefore, [0050] 

(I picture) Ti-max= MAX [A and B] A =Rav-max /(l+Nn-x^xi-Kf)) +Nb-xb/(xi-Kb)) B = BitRateMax/(8 
and PictureRate) 

(9) [0051] 

(P picture) Tp-max = MAX [C and D] C =Rav-max /(N P+ Nb-K P -xb /(«b-x P )) D = BitRateMax/(8 and 
PictureRate) 

(10) [0052] 

(B picture) Tb-max = MAX [E and F] E = Rav-max/(N P+ N P -Kb-x P /(K P -Kb)) F = BitRateMax/(8 and 
PictureRate) 

(1 1) [0053]Ti-max of an upper type, Tp-max, and Tb-max, Namely, it is a maximum of the 
target assignment code amount in the 2nd code-quantity-control means, These values are sent 
to the VBR code-quantity-control machine 51 , about the picture type concerned of the above 
mentioned picture of the present when it codes, a limiting circuit is applied with the value of Ti- 
max, Tp-max, and Tb-max, and the target assignment code amount of the present picture 
determines the value of above mentioned Ti, Tp, and Tb. 

[0054]Based on the target assignment code amount determined as mentioned above and the 
generated code amount of each macro block detected with the buffer 16, the method of MPEG 
2 Test Model 5 is used, and the quantizing scale of each macro block is determined. 
[0055]From the picture characteristic detector 25, the activity ACTcur of each macro block is 
sent also to the code-quantity-control machine 51 , Although used for the adaptive-quantization 
control which changes the quantizing scale of each macro block based on the activity in MPEG 
2 Test Model 5, it is not necessary to perform this adaptive-quantization control. The 
quantizing scale of each macro block may be determined by a completely different method 
from this. 

[0056]The quantizing scale of each macro block outputted from the code-quantity-control 
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machine 51 is sent to the quantizer 13, Numerals are outputted, after variable length coding of 
the present picture (the divided original image after DCT or the error image block of motion 
compensation prediction) is quantized and carried out with this quantizing scale and it is 
adjusted with the buffer 16. 

[0057]The generated code amount supervised with the quantizing scale for every macro block 
and the buffer 16 is sent to the normal child-ized scale detector 22 and the generated code 
amount detector 23, and is used for the code quantity control of the following picture, 
respectively. 

[0058]ln the upper explanation, amount Rav-max of average allocation codes of 1GOP in the 
CBR code-quantity-control machine 52 was calculated as a code amount to which it is simply 
assigned by 1GOP at the time of the highest transfer rate (BitRateMax).On the other hand, if 
presumed picture complexity Xk-c of a predetermined function like drawing 2 (a) which makes 
a factor presumed picture complexity Xk-c (k= i, p, b) of the present picture to be coded from 
now on, for example, the present picture, increases, the value will set up the function f (Xk-c) 
which approaches 1 infinite. 

[0059]By using Rav-max' of the following formula (12) which carried out the multiplication of 
this function for every picture type instead of Rav-max, As shown in drawing 2 (b), press down 
gradually the generated code amount in the case of being close to the highest transfer rate of 
each picture type, and. The image quality deterioration of the picture type concerned produced 
from the relation between picture complexity and the amount of allocation codes becoming 
discontinuous at the point beyond the highest transfer rate becomes possible [ also pressing 
down the problem which becomes remarkable }. 
[0060] 

Rav-max' - =f(Xk-c), (BitRateMax/PictureRate), andN -- the function (k=i, p, b) (12) which 
shows drawing 2 (a) f (Xk-c) here[0061]The 2nd working example of the video coding 
equipment of this invention is described with drawing 3 below. In the 2nd working example, it is 
a case where this invention is applied to the Variable Bit Rate code quantity control of a two 
pass method. The fundamental coding portion until it is changed into numerals with an original 
image input - the variable-length-coding machine 15 is the same as that of the 1st working 
example. Since encoding operation is performed twice (or more than it), temporary coding is 
performed first and the 2nd coding is performed based on the result of the generated code 
amount about one picture, it differs greatly that there is a portion which is different in both 
encoding operation. 

[0062]ln the 1st coding, the quantizing scale sent to the quantizer 13 is not sent from the VBR 
code-quantity-control machine 51 , but a fixed value (values, such as 6 and 8) is sent from the 
temporary coding quantization set scale machine 56 via switch SW1 , and, thereby, 
quantization of a fixed value is performed. And the bit stream after variable length coding was 
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performed in the variable-length-coding machine 15 is not sent to the buffer 16 for outputting it 
outside, but it is sent to the temporary coding generated code amount detector 53 via switch 
SW2, and the generated code amount of each picture in the 1st coding is detected. 
[0063]A temporary transfer rate is computed by a generated code amount being sent to the 
temporary transfer rate memory 54 from the temporary coding generated code amount 
detector 53 one by one, and being added for every prescribed period. This operation is 
performed until coding of one image sequence is completed, and the temporary transfer rate 
for every prescribed period is accumulated in the temporary transfer rate memory 54. 
[0064]After the 1st coding is completed, the temporary generated code amount or average 
temporary transfer rate of the whole image sequence is computed, this value and the 
temporary transfer rate for every prescribed period are sent to the target transfer rate 
calculation machine 55, and the target transfer rate for every [ in the 2nd coding (actual-code- 
izing) ] prescribed period is computed. 

[0065]The relation of the temporary transfer rate Rt for every prescribed period and target 
transfer rate R of the 2nd coding in the 1st coding sets up the predetermined function 
beforehand. For example, the following function as shown in (13) can be considered. 
[0066] 

R=a- (Rt) (13) b (a and b are a constant, a> 0, and 0< b<1)[0067]The 1st temporary coding is 
completed, and if the target transfer rate of the coding which is the 2nd time is determined, the 
2nd coding (actual-code-izing) will be started according to the target transfer rate. In the 2nd 
coding, the value from which the quantizing scale sent to the quantizer 13 was obtained with 
the VBR code-quantity-control machine 51 is sent. 

[0068]The normal child-ized scale of each frame detected with the VBR code-quantity-control 
machine 51 here with the normal child-ized scale detector 22 and the generated code amount 
detector 23, and a generated code amount, From the target transfer rate for every prescribed 
period computed from the temporary encoded result with the target transfer rate calculation 
machine 55, the target assignment code amount of the picture to be coded from now on is 
calculated. 

[0069]On the other hand, it is inputted into the VBR code-quantity-control machine 51, a 
limiting circuit is applied to said target assignment code amount, and a target assignment code 
amount also determines the maximum of the target assignment (it can set for 2nd code- 
quantity-control means) code amount computed with the CBR code-quantity-control machine 
52. The maximum of the target assignment code amount in the CBR code-quantity-control 
machine 52 is the same as that of (based on Rav-max) Ti-max in the 1st working example, Tp- 
max, and Tb-max. 

[0070]The quantizing scale of each macro block is determined using the method of MPEG 2 
Test Model 5 grade like the 1st working example based on the target assignment code amount 
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determined as mentioned above and the generated code amount of each macro block 
detected with the buffer 16. 

[0071]Thus, the quantizing scale of each determined macro block is sent to the quantizer 13, 
and variable length coding of the picture (the divided original image after DCT or the error 
image block of motion compensation prediction) to be coded from now on is quantized and 
carried out with this quantizing scale. 

[0072]Numerals are outputted after being adjusted by the target transfer rate for every 
prescribed period which the bit stream generated here was supplied to the buffer 16 by the 2nd 
coding, and was computed with the target transfer rate calculation machine 55 here. The 
generated code amount supervised with the quantizing scale for every macro block and the 
buffer 16 is sent to the normal child-ized scale detector 22 and the generated code amount 
detector 23, and is used for the code quantity control of the following picture, respectively. 
[0073]The 3rd working example of the video coding equipment of this invention is described 
with drawing 4 and drawing 5 below. Although the above working example [ 1st and 2nd ] is 
the cases where this invention is applied to Variable Bit Rate code quantity control, this 
invention is not limited to it but broad application is possible for it. Like the video coding 
equipment shown in drawing 4 , the encoded bit streams of the output of the buffer 16 are 
divided into two with the stream separator 59. The output of one of these outputs the whole 
encoded bit streams which use I like drawing 5 , P, and all B pictures, and another side outputs 
the encoded bit streams which use only I and P picture before long. 
[0074]The output of only I and P picture shall be a header conversion machine which is not 
illustrated here, and the parameter of a header part, etc. shall be rewritten by the suitable 
value. The code-quantity-control machine 1 (51 A) of drawing 4 controls the code amount of the 
whole bit stream, and the code-quantity-control machine 2 (52A) controls the code amount of 
the bit stream of only I and P picture. 

[0075]Here, both average bit rate shall be set up so that the amount of average allocation 
codes of each picture type may serve as a comparatively near value with the code-quantity- 
control machine 1 (51 A) and the code-quantity-control machine 2 (52A). In order to satisfy the 
code quantity control of two bit streams simultaneously, the code amount quota result of I in 
the code-quantity-control machine 2 (52A) and P picture is sent to the code-quantity-control 
machine 1 (51A), With the code-quantity-control machine 1 (51A), to I and P picture, I as a 
result of the code-quantity-control machine 2 (52A) and code amount assignment of P picture 
are applied as it is, code amount assignment is newly performed about B picture, and control 
of the code-quantity-control machine 1 (51 A) is realized. 

[0076]Thereby, by the whole bit stream, while control with the code-quantity-control machine 1 
(51 A) is performed, when I of a bit stream and P picture portion are taken out with the stream 
separator 59, the bit stream controlled by the code-quantity-control machine 2 (52A) can be 
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obtained. 

[0077] When the bit stream of only I and P picture does not need to be outputted with the 
stream separator 59, in the code-quantity-control machine 1 (51 A), the usual code amount 
assignment is performed about I, P, and B pictures of each, without performing code amount 
assignment with the code-quantity-control machine 2 (52A). When not performing code 
amount assignment with the code-quantity-control machine 2 (52A), the signal which shows it 
is sent to the stream separator 59, and an output is stopped for the bit stream of only I and P 
picture there. 

[0078]ln the code-quantity-control gestalt in which not only working example of drawing 4 but 
this invention has two code-quantity-control machines, as opposed to the code-quantity-control 
machine 51 A of a main direction - ** ~ by adding restriction by the code amount quota result 
for every picture type of the code-quantity-control machine 52A of a direction, Rationalization 
of code amount assignment of each picture type can be attained, or the video coding 
equipment corresponding to two decoding systems can be realized to one encoded bit 
streams. 



[Translation done.] 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1 .This document has been translated by computer. So the translation may not reflect the 
original precisely. 

2.**** shows the word which can not be translated. 
3.ln the drawings, any words are not translated. 



DESCRIPTION OF DRAWINGS 

[Brief Description of the Drawings] 

[Drawing 1] lt is a block lineblock diagram showing the video coding equipment of this 
invention, and the 1st working example of the method. 

[Drawing 2] lt is a figure showing the relation between the function in the case of Rav-max' 
calculation of the 1st working example of this invention, and Xk-c and the amount of allocation 
codes. 

[Drawing 3] It is a block lineblock diagram showing the video coding equipment of this 
invention, and the 2nd working example of the method. 

[Drawing 4] lt is a block lineblock diagram showing the video coding equipment of this 
invention, and the 3rd working example of the method. 

[Drawing 5l lt is a figure showing the situation of the bit stream division by the stream division 

means of the 3rd working example of this invention. 

[Drawing 6] lt is a figure showing an example of coding picture structure. 

[Drawing 7] lt is a figure showing the example of 1 composition of common video coding 

equipment. 

[Drawing 8] It is a figure showing the example of 1 composition of conventional video coding 
equipment. 

[Description of Notations] 

11 Subtractor 

12 DCT device (DC to AC converter) 

13 Quantizer 

14 Code-quantity-control machine 

15 Variable-length-coding machine 

16 Buffer 

17 Inverse quantization device 



http://www4.ipdl.inpit.go.jp/cgi-bin/tran_web_cgi_ejje?atw_u=http7o3A7o2... 5/13/2010 



JP,2001-028753,A [DESCRIPTION OF DRAWINGS] 



Page 2 of 2 



18 IDCT machine 

19 Motion-compensation-prediction machine 

20 Adding machine 

21 Frame memory 

22 Normal child-ized scale detector 

23 Generated code amount detector 

24 Picture complexity calculation machine 

25 Picture characteristic detector 

51 VBR code-quantity-control machine 

52 CBR code-quantity-control machine 
51 A Code-quantity-control machine 1 
52A Code-quantity-control machine 2 

53 Temporary coding generated code amount detector 

54 Temporary transfer rate memory 

55 Target transfer rate calculation machine 

56 Temporary coding quantization set scale machine 
59 Stream separator 

The amount of average allocation codes of 1GOP in case Rav-max BitRate is the highest 
transfer rate (BitRateMax) 

The amount of required allocation codes of Rck 1GOP (k= i, p, b) 
SW1 and SW2 Switch 

The target assignment code amount of Tk each picture type (k= i, p, b) 

The target assignment code amount of each picture type in case Tk-max BitRate is the highest 

transfer rate (BitRateMax) (k= i, p, b) 

Picture complexity of the picture of Xk present (k= i, p, b) 

Presumed picture complexity of the picture of the Xk-c present (k= i, p, b) 



[Translation done.] 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1 This document has been translated by computer. So the translation may not reflect the 
original precisely. 

2.**** shows the word which can not be translated. 
3.ln the drawings, any words are not translated. 
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o xk-c Kttt&mmsHk Tk(k-i, p, b)<om* 



Rav-max Tk-max ( Rav max = (BitRateMax/PicturoRata) • N ) 




iv-max* t= X i\ Tk-max ( Rav-max' = f (Xk-c) - Rav-max ) 



w<m», xy f xk-ctsi^#t^Tk(k=i,p,b)oH#w« 



[Drawing 3] 
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[Drawing 5] 
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[Drawing 4] 
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[Drawing 7] 
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[Drawing 81 
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[Translation done.] 
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